1. Field of the Invention
The present inventive concept relates to a camera system, comprising at least one image sensor based camera unit, attached to the vehicle in order to improve the safety of driving, and a method for displaying anticipated trajectory of a vehicle and a system for generating driving corridor markers.
2. Description of the Related Art
Image sensor based cameras are becoming widely used in the vehicles. By capturing a portion of the environment surrounding the car and displaying it on the monitor in front of driver, the system enables better sense of the location and orientation of the vehicle with respect to the other objects (other vehicles, pedestrians, cyclists, buildings, trees, road and parking lot marking, etc.), improves control over the traffic situation, thus improving the safety of driving. In particular, as illustrated in FIG. 1, the camera attached to the rear part of the vehicle (rear view camera) captures the area behind the vehicle, which normally cannot be seen from the driver's seat. Likewise, the front view camera may be attached to the front part of the vehicle, and captures the area in front of the vehicle.
The camera unit is preferably small enough to be placed in the rear panel boot lid or grill of the vehicle without spoiling the appearance of the vehicle.
In order to maximize the area captured by the camera and displayed on the monitor, the rear-view cameras incorporate the lenses with wide field of view (FOV), typically 120-135°, in some cases up to 190°. The disadvantage of such cameras is, that due to the difference between the FOV of the camera and average FOV of human vision, the distant objects captured by the camera appear smaller compared to the near objects, than a human would expect. This disproportion may cause confusion: it becomes hard to estimate the dimensions of the driven car with respect to other objects shown on the captured image.
In order to overcome this problem, as shown in FIG. 2, markers depicting the dimensions of the vehicle against the object distance are displayed along with the image captured by the camera. As soon as the markers do not change according to the steering angle, this kind of displayed markers is often referred to as “static overlay.”
In order to make driving even safer and easier, especially when maneuvering in close space, like garage, or parking lot, the camera systems may feature display of driving corridor, often referred to as “dynamic overlay.” This function is similar to static overlay, except the markers depicting the dimensions of the vehicle change according to the steering angle. Thus, the driving corridor displays the suppositional trajectory of the vehicle with respect to the surrounding objects, assuming the vehicle is driven with the current steering angle. If the driver changes the steering angle by turning the steering wheel, the displayed driving corridor changes accordingly (FIG. 3).
In order to implement such functionality, the camera system has to communicate with other devices of the vehicle, for example steering wheel, typically via controller area network (CAN) bus or local interconnect network (LIN) bus, in order to acquire the descriptor of steering angle, although the other interfaces may be used. Therefore, the camera system incorporates an interface transceiver and interface controller. The interface controller usually incorporates a microprocessor with the processing power just enough to support input/output functions and basic calculus, random access memory (RAM) for program and data, and non-volatile (ROM, flash) memory to store the program code and data (parameters).
In order to produce the displayed image with dynamic overlay, the camera system incorporates image processing device. This device inputs the image captured by image sensor, adds the overlay to be displayed, and transmits the resulting image to the monitor, typically as analog signal, NTSC or PAL.
In order to generate the overlay image, the image processing device inputs the descriptor of steering angle from the interface controller via I2C or any other available interface. Then, according to the steering angle, the location of the markers depicting driving corridor on the image, is calculated by image processing device and superimposed on the image acquired from the image sensor.
Typical frame of the video displayed on the monitor, contains about 350,000 elements (pixels). Therefore, the image processing device must be capable of processing this number of elements with the required frame rate, usually about 30 frames per second.
At the moment, there are three classes of devices that possess sufficient processing power: Digital Signal Processor (DSP), Field Programmable Gate Array (FPGA), Complex Programmable Logic Device (CPLD), and Application Specific Integrated Circuits (ASIC).
An example of known state of the art architecture of the camera system is shown in the FIG. 4. The system comprises two parts, i.e. a camera unit 100 and an electronic control unit (ECU) 200.
ECU 200 comprises an image processing device (DSP, FPGA CPLD or ASIC) 204, an interface transceiver 201, an interface controller 202, an analog decoder 203 and an analog signal encoder 205.
Camera unit 100 outputs the image obtained by an image sensor 101 as an analog (typically NTSC/PAL) signal, which is transmitted to ECU 200 via an analog signal encoder 102.
The analog signal decoder 203 on ECU 200 converts the signal into digital signal, which is acquired by the image processing device 204. Steering angle descriptor is acquired by the interface transceiver 201 and transmitted to the interface controller 202. The interface controller 202, in turn, transmits the corresponding data to the image processing device 204. Based on that data, the image processing device 204 generates corresponding driving corridor markers, superimposes them on the image acquired from the camera unit 100 separated from ECU 200, and transmits the resulting image to the analog signal encoder 205, which converts the image into analog signal and transmits it to the monitor.
One of the commonly used state of the art methods to generate the driving corridor markers is as follows (FIG. 5). The bitmap depicting the overlay, usually one per each steering angle value, are stored in the non-volatile memory in ECU 200. The image processing device 204 obtains the steering angle value from the interface controller 202, downloads the corresponding bitmap from the non-volatile memory into RAM and superimposes it on the image acquired from the image sensor 101 in the separate camera unit 100.
Note, that the amount of non-volatile memory required to store overlaid bitmaps covering the whole range of steering angles is relatively large, thus increasing the overall size and cost of the system.
The main disadvantages of this conventional architecture are as follows:
1. Camera system comprises at least two separate devices (camera unit and ECU), because, due to the limitations of camera size, it is not possible to embed the image processing device with sufficient processing power and/or sufficiently large non-volatile memory to store overlaid bitmaps into the camera unit. The use of multiple devices results in higher system complexity, higher cost, overall dimensions, weight, etc.
2. The image acquired by the camera unit is transmitted to ECU over analog signal, because due to significant distance between the camera unit and ECU state of the art digital interfaces are not sufficiently robust against noise, require more than two wires or have insufficient bit rate. However, due to multiple digital-to-analog and analog-to-digital conversions required for analog interface, the image quality degrades significantly.